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PREFACE 


This  final  technical  report  covers  the  research  performed 
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and  Astronautics ,  Measurement  Systems  Laboratory  and  Man-Vehicle 
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SUMMARY 


The  major  result  of  this  work  has  been  to  quantify  on  a 
statistical  basis  the  variations  in  skill  among  non-instrument 
rated  private  and  commercial  pilots.  Variations  between  pilots 
of  the  population  due  to  individual  differences  in  their  latent 
skill  accounts  for  30%  of  the  observed  variance.  Variations 
due  to  quantitative  experience  factors  accounts  for  25%  of  the 
variance.  The  remaining  45%  is  due  to  random  variations  of  in¬ 
dividual  pilots ,  interaction  effects  and  noise  in  the  measure¬ 
ment  process. 

The  variations  due  to  experience  factors  can  be  represen¬ 
ted  by  a  regression  model.  This  is  an  equation  which  predicts 
average  skill  icore  based  on  Recency  in  type,  logarithm  of  the 
total  time,  years  since  certification,  and  logarithm  of  the 
time  in  type.  Each  of  these  factors  contributes  linearly  to 
the  overall  skill  score  in  an  additive  way.  The  equation  which 
relates  these  experience  factors  to  skill  is  based  on  an  obser¬ 
vation  of  the  pilot  population.  It  has  not  been  shown  that  a 
change  in  these  factors  causes  a  change  in  the  skill  of  an  in¬ 
dividual  pilot.  However,  it  is  probably  reasonable  to  assume 
that  it  does,  and  this  assumption  is  required  in  order  to  apply 
the  equation  in  practise.  The  equation  tells  us  that  pilots 
who  have  either; 

.4  hrs/week  more  type  recency,  or 
32%  more  total  time,  or 
3.3  fewer  years  since  certification,  or 
150%  more  time  in  type, 

have  on  the  average  a  0.1  skill  unit  higher  overall  grade. 

(The  standard  deviation  of  the  overall  grade  was  0.7  skill 
units . ) 

The  most  important  of  these  experience  factors  was  Recency, 
It  accounted  for  the  largest  percentage  of  the  variance  (40% 
of  the  contribution  of  all  the  experience  factors  combined) . 

It  is  the  experience  factor  which  can  be  varied  most  easily. 

(A  single  four  hour  flight  will  increase  Recency  by  one  hour 
per  week.  Recency  wilx  decay  exponentially  to  zero  with  a  time 
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constant  of  four  weeks  with  no  flying.)  The  second  most  im¬ 
portant  experience  factor  was  the  logarithm  of  the  total  time. 

It  accounted  for  the  next  largest  percentage  of  the  variance 
(28%  of  the  total  for  all  experience  factors) .  The  loaarithmic 
dependence  causes  changes  in  total  time  to  be  more  important 
for  pilots  with  low  total  time.  These  results  are  valuable 
for  helping  pilots  to  appreciate  the  importance  of  totvl  and 
recent  experience. 

The  results  of  the  study  are  also  pertinent  to  the  analysis 
of  recent  changes  in  Part  61  of  the  Federal  Air  Regulations.  Con¬ 
sider  the  requirement  for  a  periodic  flight  review  on  the  basis  of 
a  linear  decrease  in  skill  with  years  since  certification.  The 
current  average  for  this  population  is  about  six  years  since 
certification.  Make  the  assumption  that  a  biennal  flight  review 
will  reduce  the  effective  average  to  near  one  year.  The  five 
year  reduction  accounts  for  a  .15  average  skill  unit  increase. 
Making  the  flight  review  an  annual  requirement  would  only  in¬ 
crease  the  average  skill  by  an  additional  .015.  It  appears 
that  the  biennal  review  is  helpful  but  that  requiring  it  more 
often  than  every  2  years  would  result  in  diminishing  returns. 
Pilots  do  not  like  to  take  check  rides,  but  such  flights  can  be 
effective  in  improving  skill.  Anticipation  of  the  flight  review 
should  cause  pilots  to  schedule  preparatory  flights  -  thus 
increaaing  their  recency,  if  nothing  else. 

Concerning  the  reduction  in  the  requirement  for  landings  in 
the  last  90  days  from  five  to  three,  the  following  applies: 

This  study  suggests  recent  flight  experience  to  be  the  most 
important  experience  factor  considered  to  ..explain  observed 
skill.  The  effect  of  experience  90  days  old  on  Recency  as 
defined  in  this  study  is  near  zero.  Furthermore,  three 
landings  per  90  days  correlates  with  a  flight  time  of  only  1.5 
hours  in  90  days.  Thif?  amount  of  flight  time  would  h*ve  a 
negligible  effect  on  average  skill. 

In  order  to  bring  a  pilot  with  degraded  skills  back  up  to 
standard,  it  is  necessary  to  increase  the  recency  in  proportion 
to  tho  skill  degradation.  The  ratio  of  the  two  suggests  that 


.15  hour/week  recency  is  required  to  offset  the  effect  of  each 
year  idle.  If  this  recency  is  acquired  all  at  one  time,  then 
the  requalification  of  a  degraded  pilot  should  require  0.6 
hours  of  flight  for  each  year  spent  with  no  flying.  Although 
flying  skills  degrade  rapidly  when  not  used,  they  return  rapidly 
with  practice. 

Individual  differences  in  latent  skill  are  more  important 
than  experience  factors  in  predicting  overall  skill  scores. 

The  results  of  written  quizzes  are  not  effective  in  predicting 
flight  test  results. 
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I  INTRODUCTION 


The  purpose  of  this  work  was  to  measure  skill  degradation 
of  non-instrument  rated,  single-engine,  FAA  certificated  private 
and  commercial  pilots.  Selectees  from  the  pilot  population  were 
tested  to  determine  the  measurable  and  recent  experience  factors 
which  are  important  for  the  maintenance  of  aeronautical  skills 
within  this  profile.  The  program  was  designed  to  measure  the 
statistical  relationship  between  skill  and  specific  experience 
factors . 

The  sample  for  testing  was  selected  from  the  total  popula¬ 
tion  of  FAA  certificated  private  and  commercial  pilots  who  were 
single-engine  airplane  rated. 

A  mailing  list  of  one  thousand  pilots  was  assembled  from 
the  FAA  Airmen  Directory  File  by  selecting  every  10th  pilot 
from  those  pilots  with  addresses  in  the  New  England  states. 
Letters  (Appendix  A-l)  were/  mailed  to  this  group  inviting  them 
to  participate  in  the  flight  test  program.  The  statistical 
characteristics  of  the  pilots  on  the  mailing  list  was  checked 
against  the  statistical  characteristics  known  about  the  total 
pilot  population  (References  1  and  2)  to  ensure  the  list  was 
representative . 

A  sample  of  seventy-one  pilots  was  selected  from  those  324 
who  returned  the  Pilots'  History  Questionnaire  (Appendix  A-2) 
and  indicated  willingness  to  .participate  in  the  flight  test 
program.  The  statistical  properties  of  the  sample  of  the  71 
pilots  were  also  checked  against  characteristics  known  about 
the  total  pilot  population  (References  1  and  2) .  In  addition, 
the  returned  questionnaires  provided  a  measure  of  actual  flight 
recency  and  the  pilots'  opinions  concerning  their  own  skill 
degradation. 

An  objective  type  flight  test  was  adapted  from  an  Embry 
Riddle  Study  described  in  Reference  3.  The  flight  test  proce¬ 
dure  was  the  same,  but  the  grading  was  modified  in  that  a 
numerical  grade  on  the  various  phases  of  each  flight  was  sub¬ 
stituted  for  the  long  binary  check  list  previously  used. 
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Flights  were  conducted  in  a  Cessna  150  based  at  the  M.I.T.,  C.S. 
Draper  Laboratory  Flight  Facility  at  Hanscom  Field,  Bedford, 
Massachusetts.  The  testing  was  conducted  by  five  instructor 
pilots  from  M.l.T. 

The  standard  program  was  as  follows: 

1st  meeting:  -Brief  on  program 

-Complete  subjects'  Experience  Data  Form 
-Short  written  quiz  on  aeronautical  knowledge 
-First  flight  test  of  aeronautical  skills 
(Basic  Piloting) 

2nd  meeting:  -Short  written  quiz  on  aeronautical  knowledge 
-Plan  cross-country  flight  to  destination 
and  return 

-File  VFR  flight  plan 
-Conduct  cross-country  flight 

3rd  meeting:  -Normally  one  month  after  second  flight 

-Test  aeronautical  skills  (repeat  first  flight) 
-Review  evaluation  of  aeronautical  skills  and 
knowledge  with  subjects  tested 

Each  maneuver  or  phase  of  flight  to  be  evaluated  was  iden¬ 
tified  by  the  type  of  skill  that  was  primarily  involved *  The 
skill  types  considered  were: 

Procedural  Skills  (retention  and  recall) 

Judgement  and  Problem  Solving 
Motor  Coordination 

Each  subject's  performance  on  each  phase  of  each  flight 
was  graded  on  a  continuous  scale  from  0-5  which  had  the  follow¬ 
ing  interpretation: 

5  perfect 
4  above  average 
3  average 
2  below  average 
1  unacceptable 
0  dangerous 
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Each  subject  was  also  given  one  score  that  represented  the 
examiner ' s  evaluation  of  the  subject's  over-all  aeronautical 
skill  and  knowledge. 

The  record  of  all  scores  has  been  kept  separate  from  the 
names  of  the  subjects  for  confidentiality. 

Data  processing  was  performed  to  show  statistically  from 
the  measurements  how  skill  varied  as  a  function  of  the  follow¬ 
ing  factors  known  about  each  subject: 

-Total  Plight  Time 
-Recency  (current  hrs/week) 

-Time  in  Type  (Cessna  150) 

-Time  since  Certification  (years) 

-Type  Initial  Training  (Military,  FAA 
Approved  School,  etc.) 

-Type  of  Flight  Operations  Conducted  (Con¬ 
trolled  Air  Field  vs.  Uncontrolled  Air 
Field) 

-Age 

-Gender 

-Geographic  Location  of  majority  of  flights 


3 


II  THEORY 


The  variations  in  observed  piloting  skill  can  be  considered 
to  be  due  to  several  causes  classified  as  follows: 

1.  Variation  due  to  the  fact  that  the  latent  skill  of 
each  individual  is  different ,  independent  of  previous 
experience. 

2.  Variation  due  to  quantitative  factors  which  describe 
the  subjects'  experience,  such  as  Total  Time,  Recency, 
Age,  Years  Since  Certification,  etc. 

3.  Variation  in  measured  skill  due  to  a)  the  observer, 
b)  the  measurement  process,  or  c)  interaction  effects 
between  experience,  observer,  and  measurement  factors. 

4.  Random  variations  in  the  demonstrated  skill  of  an  in¬ 
dividual  subject  not  accounted  for  by  the  foregoing. 

Most  flight  checks  have  the  purpose  of  evaluating  overall 
skill  without  attempting  to  differentiate  among  these  four 
groups.  In  this  study,  the  emphasis  has  been  placed  on  measur¬ 
ing  skill  variations  due  to  factors  in  the  second  group.  A 
major  problem  in  such  testing,  however  is  that  the  sources  of 
variation  due  to  the  other,  primarily  hon-experience  related 
groups  have  been  found  by  this  study  to  be  at  least  as  important. 
Thus,  although  the  effects  of  the  experience  factors  have  been 
measured,  and  can  be  considered  known  with  a  reasonable  degree 
of  statistical  reliability,  at  least  for  the  test  population, 
knowledge  of  second  group  factors  is  of  only  limited  value  in 
predicting  overall  skill  due  to  variation  introduced  by  other 
components.  For  example,  statistical  analysis  of  the  test  data 
in  this  study  indicates  that  the  first  group  of  factors  accounts 
for  about  30%  of  the  variance  in  the  data.  The  second  group  of 
experience  factors  accounts  for  about  25%.  The  remaining  45% 
is  due  to  groups  3  and  4. 


* 


Measurement  of  Experience  Factors 


Certain  aspects  of  an  individual  pilot's  experience  can 
be  easily  represented  directly  by  single  parametric  variables. 
Examples  include  Total  Time,  Total  Time  in  Type,  Age,  Years 
Since  Certification,  etc.  However,  development  of  a  quantita¬ 
tive  parameter  which  summarizes  an  individual  pilot's  recent 
flight  experience  in  a  reasonable  way  is  a  more  difficult 
problem.  Since  it  was  expected  that  the  nature  of  an  individ¬ 
ual's  recent  flight  experience  would  heavily  influence  his 
aeronautical  skills,  it  was  decided  that  an  attempt  at  para¬ 
metric  representation  of  recent  flight  history  should  be  made, 
for  the  purpose  of  this  study. 

The  method  chosen  was  to  define  a  parameter  called  "recency” 
which  can  be  calculated  for  an  individual  subject  given  his 
racent  flight  history «  Actually,  four  different  kinds  of 
recency  were  defined: 

Total  Recency  (hrs/week) 

Type  (Cessna  150)  Recency  (hrs/week) 

Landing  Recency  (landings/week) 

Type  (Cessna  150)  Landing  Recency  (landings/week) 

These  "recency"  parameters  were  constructed  mathematically 
so  as  to  have  the  following  useful  properties: 

1.  For  a  subject  who  practiced  regularly  at  a  constant 
rate,  his  values  of  recencies  would  be  equal  to  his 
practice  rates. 

2.  If  the  subject  stopped  flying  altogether,  his  values 
of  recency  would  decay  away  exponentially  to  zero  over 
a  characteristic  degradation  time. 

(Total  Recency  can  be  conceptualized  as  the  result  of 
passing  the  instantaneous  rate  of  change  of  total  time  through 
a  low  pass  filter  with  a  time  constant  equal  to  a  characteristic 
skill  degradation  time.) 

A  mathematical  approach  to  the  recency  concept  is  given 
in  Appendix  B. 
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SUBJECT:  DATES  »  JULIAN 
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• 

• 
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Y 
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6 
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1.0 
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ft 

• 

2 
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N 

FLIGHT 

as 

5 
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177 

DUR 

0.8 

# 
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• 

• 

5 

C-150? 

N 

FLIGHT 

rx 

4 

DATE 

183 

DUR 

0.7 

# 

LNDGS 

* 

• 

2 

C-150? 

N 

FLIGHT 

as 

3 

DATE 

184 

DUR 

0.7 

# 
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• 

• 

2 
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N 

FLIGHT 

S3 

2 

DATE 

207 

DUR 

0.7 

# 
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• 

• 

2 

C-150? 

N 

FLIGHT 

St 

1 

DATE 

223 

DUR 

0.8 

# 

LNDGS 

• 

• 

1 

C-150? 

Y 

TEST  FLIGHT 

*  1 

DATE 

:  267 

DUR 

:  1.6 

* 

LNDGS  : 

5 

TOTAL  TIME  =  258 

TOTAL  C-150  TIME  =  54 

RECENCY  »  0.10 

LANDING  RECENCY  =  0.23 

C-150  RECENCY  =  0.04 

C-150  LNDG  REC.  =  0.06 

#  OF  INTERFLIGHTS  :  1 
INTERFLIGHT  #  =  1 

DATE  :  271  DUR  :  1.75  #  LNDGS : 2  C-150? :Y 

TEST  FLIGHT  *  2 

DATE  :  295  DUR  :  2.0  #  LNDGS  :  2 

TOTAL  TIME  *  261 

TOTAL  C-150  TIME  *  57 

RECENCY  *  0.37 

LANDING  RECENCY  =  0.76 

C-150  RECENCY  *  0.35 

C-150  LNDG  REC.  *  0.69 

#  OF  INTERFLIGHTS  :  0 
TEST  FLIGHTS  «  3 

DATE  :  309  DUR  :  1.1  #  LNDGS  :  5 

TOTAL  TIME  *  263 

TOTAL  C-150  TIME  -  59 

RECENCY  *  0.53 

LANDING  RECENCY  *  0.76 

C-150  RECENCY  »  0.52 

C-150  LNDG  REC.  *  0.72 

EXAMPLE  RECENCY  CALCULATION 

Figure  1. 
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An  example  showing  how  an  individual's  values  of  recency 
are  determined  by  his  recent  flight  history  is  shown  in  Figure  1 
(see  also  section  IV:  Analysis) . 

The  general  approach  used  in  the  data  analysis  was  as 
follows:  Given  a  parametric  representation  of  a  pilot's  skill 
(his  overall  flight  check  grade) ,  a  multidimensional  regression 
analysis  was  used  to  determine  the  effects  of  "group  2"  expe- 
rience  factors ,  including  recencies ,  on  the  mean  grade  of  the 
sampled  population.  Theoretically ,  it  might  haveNbeen  possible 
to  analytically  estimate  an  "optimal"  value  for  the\pharacter- 
istic  degradation  time  constant  of  recency  by  iteratively 
fitting  a  regression  model  for  skill  to  different  sets  ^f  data 


generated  by  assuming  different  values  of  recency  time  constant. 


and  defining  the  "optimal"  time  constant  to  be  that  which  mini¬ 
mized  the  variance  in  the  overall  model.  However,  such  an 
approach  was  not  chosen  because  the  quality  of  the  final  re¬ 
gression  fit  was  not  expected  to  vary  significantly  for  large 
changes  in  the  recency  time  constant.  This  was  because  many  of 
our  pilots  flew  regularly  (for  these  individuals,  the  assumption 
of  a  particular  time  constant  does  not  affect  the  value  of  cal¬ 
culated  recency) .  Also ,  in  most  cases ,  the  testing  program  was 
unable  to  significantly  perturb  the  recency  parameters  of  in¬ 
dividual  pilots  unless  the  time  constant  was  assumed  to  have 
unrealistically  small  values.  Given  this  situation,  the  overall 
objectives  of  the  study,  and  the  order  of  magnitude  increase  in 
complexity  of  the  data  analysis  which  such  a  search  procedure 
would  require,  an  alternative  method  for  estimation  of  the  re¬ 
cency  time  constant  was  adopted.  This  consisted  of  basing  our 
recency  calculations  on  an  estimate  of  the  time  constant  fur¬ 
nished  by  the  pilots  themselves.  This  estimate  was  obtained  by 
taking  the  answer  to  a  carefully  chosen  question  asked  of  all 
subjects  tested  as  well  as  all  those  contacted  by  mail  in  the 
original  survey  of  the  New  England  region: 

"Assume  that  you  were  flying  regularly  at  the  rate  of  one 
hour  per  week  and  that  you  stopped  all  flying  for  a  period 
of  six  months .  How  many  flight  hours  would  you  estimate 
you  would  need  in  order  to  regain  your  former  pilot  profi¬ 
ciency?" 


\ 
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The  answer  to  the  question  in  hours  is  a  measure  of  the  sub¬ 
ject's  estimate  of  his  own  characteristic  degradation  time  (t) 
in  weekt\.  The  median  value  of  t  from  this  measure  was  three 
weeks.  The  mean  value  was  4.6  weeks.  The  distribution  was 
skewed  and  very  broad  as  shown  in  Figure  2.  As  a  result t  it 
was  decided  that  the  definition  of  Recency  would  be  made  using 
a  value  of  t  of  exactly  four  weeks  for  all  pilots. 
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BMD35D  GENERAL  PLOT  -  INCLUDING  HISTOGRAM  -  REVISED  JANUARY  30,  1S70 
HEALTH  SCIENCES  COMPUTING  FACILITY,  UCLA 
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Ill  PLIGHT  TESTING 


Plight  testing  operations  were  conducted  from  a  Plight 
Operations  Center  established  at  the  M.i.T.,  C.  Stark  Draper 
Plight  Facility  located  at  Hanscom  Field,  Bedford,  Massachusetts. 
All  flight  testing  was  done  in  a  Cessna  150  aircraft. 

Plight  testing  was  scheduled  during  the  period  September 
1972  through  March  1973,  seven  days  per  week  when  feasible. 

During  this  period  a  total  of  381  program  flights  were  scheduled. 
Flight  cancellations  due  to  weather  totaled  148  (39%)  and  52 
(14%)  for  other  reasons  (no  shows,  illness,  etc.).  In  all, 

53%  of  all  scheduled  flights  were  cancelled  for  reasons  beyond 
the  control  of  the  Plight  Operations  Center. 

Evaluators 

Five  experienced  pilots  from  M.I.T.  were  assigned  as  eval¬ 
uators  for  the  flight  testing  program.  A  breakdown  of  their 
flight  experience  is  as  follows : 

AGE  FAA  CERTIFICATE  RATINGS— FLIGHT  EXPERIENCE 

24  Commercial  Pilot  Airplane  SEL,  MEL 

Instruments 
Flight  Instructor 

Ground  Instructor  (Advance,  Instruments) 
950  Hours  Total  Time 

35  Commercial  Pilot  Airplane,  SEL 

Instruments 
Flight  Instructor 

Ground  Instructor  (Advance,  Instruments) 
1200  Hours  Total  Time 

41  Commercial  Pilot  Airplane  SEL,  MEL 

Instruments 
Flight  Instructor 
3000  Hours  Total  Time 
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AGE 

48 


Commercial  Pilot 


RATINGS— FLIGHT  EXPERIENCE 

Airplane  SEL 

Glider 

Flight  Instructor 
USN  Aircraft  Commander  MEL  and 
Instrument  Instructor 
7200  Hours  Total  Time 


53  Commercial  Pilot  Airplane  SEL,  MEL 

Rotorcraft 

Helicopter 

Glider 

Instruments 

Flight  Instructor 

Society  of  Engineering  Test  Pilots 

USAF  Command  Pilot 

11,500  Hours  Total  Time 

Evaluator  standardisation  was  achieved  through  discussion 
periods,  standardization  flights  and  the  following  guidelines: 

Skill  grades  were  assigned  as  indicated  on  the  Flight 
Evaluation  Record  form  (Figure  3)  for  major  subareas  of  each 
flight,  plus  an  overall  grade  and  written  quiz  grade  when  taken 
A  grade  was  entered  in  all  boxes  for  which  the  subjects'  per¬ 
formance  was  observed  and  a  dash  if  the  box  was  not  applicable 
to  the  flight  or  the  maneuver  was  not  -formed.  Grades  were 
assigned  on  the  basis  of  0-5  as  follows: 

5  *  perfect 
4  =  above  average 
3  *  average 
2  *  below  average 
1  =  unacceptable 
0  =  dangerous 

In  general  the  criteria  for  "average”  was  that  established 
by  the  FAA  Private  Pilot  Flight  Test  Guide  AC  61.  Grades  were 
assigned  on  observed  performance  in  the  following  three  areas: 
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1.  Procedures,  Retention  and  Recall 

The  subject  was  expected  to  be  knowledgeable  concerning 
PAR,  Part  61 — Certification:  Pilots  and  Plight  Instructors  and 
Part  91 — General  Operation  and  Flight  Rules.  Written  quizzes 
were  administered  to  each  subject  prior  to  the  first  two 
flights,  but  evaluators  were  expected  to  ask  questions  and  ob¬ 
serve  the  subjects'  adherance  to  specified  rules  and  procedures 
as  required  for  safety  of  flight. 

2 .  Judgment  and  Problem  Solving 

Grades  in  this  area  were  based  on  the  subjects'  ability  to 
use  whatever  information  was  available  to  him  and  to  apply  it 
as  would  be  expected  for  his  level  of  pilot  certification. 
Especially  important  was  the  subjects'  judgment  and  actions  as 
related  to  safety  of  flight. 

3.  Motor  Coordination 

The  "average"  pilot  subject  was  expected  to  demonstrate 
the  ability  to  maintain  the  aircraft  in  a  safe  flight  attitude 
under  all  normal  conditions.  For  all  maneuvers,  it  was  required 
that  airspeed  be  maintained  within  plus  or  minus  5  mph,  altitude 
within  plus  or  minus  100  feet  and  heading  within  plus  or  minus 
10  degrees.  In  addition,  the  subject  was  expected  to  be  able 
to  quickly  recognize  unsafe  flight  situations  and  to  take 
proper  corrective  action. 

Overall  Grade 

This  was  the  evaluation  of  the  overall  skill  and  knowledge 
demonstrated  on  each  flight  observed. 

Quiz  Grades 

Two  quizzes  were  administered  to  the  subject.  Quiz  #1 
(Appendix  C-l)  covered  general  operating  rules  and  procedures 
and  Quiz  #2  (Appendix  C-2)  covered  cross  country  flight  require¬ 
ments.  Quiz  questions  were  typical  sample  examination  questions 
from  Private  Pilots'  and  Commercial  Pilots'  Handbooks. 
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Flight  Planning  &  Filing 

The  subject  was  expected  to  file  a  VFR  flight  plan  for  a 
two  hour  cross  country  flight.  The  procedures  grade  reflected 
his  skill  in  obtaining  and  evaluating  weather,  plotting  his 
course  on  a  chart,  and  establishing  check  points  and  distances, 
estimated  time  enroute,  headings,  altitudes,  weight  and  balance, 
and  fuel  requirements.  Planning  operations  were  to  be  completed 
in  less  than  30  minutes.  The  judgment  and  problem  solving  grade 
reflected  demonstrated  judgment  in  dealing  with  special  problems 
created  by  weather,  restricted  areas,  Notams,  etc. 

Aircraft  Preflight 

The  preflight  sheck  was  expected  to  be  completed  in  accor¬ 
dance  with  the  checklist  provided  in  the  aircraft.  The  subject 
was  required  to  determine  whether  the  aircraft  was  ready  for 
flight  and  to  know  the  appropriate  remedial  action  for  the  cor¬ 
rection  of  unsatisfactory  items.  No  attempt  was  made  to  create 
discrepancies.  Below  average  grades  reflected  failure  to  check 
or  detect  important  items.  Above  average  grades  reflected 
careful  checking  or  clever  detection  of  obscure  discrepancies. 

Start,  Taxi  and  Run-Up 

This  phase  was  also  expected  to  be  in  accordance  with  the 
check-list  provided  in  the  aircraft.  Specific  items  observed 
were  clearing  the  propeller  area  before  starting  the  engine, 
use  of  the  controls  to  counter-act  wind  while  taxiing  at  a  safe 
speed,  checking  area  for  clearance  prior  to  run-up  and  adherence 
to  taxi  clearance  procedures. 

Take-Off  Departure 

The  motor  coordination  grade:  was  based  on  the  subjects' 
demonstrated  take-off  and  climb  out  performance.  The  grade  was 
based  on  an  average  of  all  the  take-off3  observed  during  the 
flight,  but  the  initial  take-off  and  climb-out  was  given  the 
greatest  weight.  The  procedural  grade  was  based  on  adherence 
to  procedures  for  departure  from  the  traffic  pattern  and  for 
take-off  and  climb  of  the  aircraft  as  appropriate.  A  below 
average  grade  was  assigned  for  any  violation  of  established 
procedures. 
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Slow  Flight  and  Stalls 

Slow  Plight  was  conducted  at  a  speed  such  that  20°  increase 
in  pitch  attitude  caused  a  stall.  In  the  Cessna  150  this  is 
at  a  speed  near  55  mph  with  the  stall  warning  audible.  Stalls 
were  entered  from  approach ,  departure  and  accelerated  flight 
conditions.  The  subject  was  expected  to  recognize  and  make  a 
normal  recovery  from  the  stalls. 

VOR  Orientation  and  Tracking 

The  procedural  grade  was  based  on  the  subjects'  ability  to 
determine  and  tune  the  VOR  to  the  desired  station  frequency, 
identify  the  station,  and  obtain  the  bearing  to  it.  The  judgment 
grade  was  based  on  the  subject's  ability  to  use  the  VOR  to 
determine  the  aircraft's  position  and  to  solve  simple  navigation 
and  tracking  problems. 

Simulated  Engine-Out 

The  simulation  of  a  failed  engine  was  initiated  by  simul-’ 
taneously  retarding  the  throttle,  adding  carburetor  heat  and 
directing  the  subject  to  proceed  as  if  the  engine  had  failed. 
This  was  done  at  least  once  each  flight  when  least  expected. 

It  was  desirable  that  the  simulated  emergency  be  carried  out 
completely  to  a  touch  down,  but  if  this  was  not  practicable  the 
latter  portion  was  simulated  from  the  traffic  pattern  around 
an  active  runway.  This  simulation  was  seldom  carried  out  com¬ 
pletely  to  a  landing.  For  all  simulated  engine  failures  the 
subject  was  expected  to  immediately  establish  a  glide,  pick 
his  spot  for  landing  and  announce  his  plan.  The  grades  reflect 
the  performance  averaged  over  all  simulated  emergencies  given 
during  the  flight.  The  judgment  grade  refers  to  the  choice  of 
field,  planning  and  approach,  accounting  for  the  wind,  and 
evaluating  the  situation.  The  coordination  grade  refers  to  how 
well  the  subject  handled  the  aircraft  in  a  power  off  condition. 
Below  average  grades  were  given  for  actions  which  would  have 
not  resulted  in  a  safe  landing. 

Simulated  Loss  of  Horizon 

This  simulation  was  conducted  during  the  cross-country 
flight  at  an  unannounced  time.  An  instrument  hood  was  handed 
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to  the  subject  and  he  was  told  to  assume  that  he  had  inadver¬ 
tently  flown  into  clouds.  He  was  expected  to  be  able  to  make 
a  180°  reversal  on  instruments  and  present  a  revised  plan  to 
cope  with  the  situation.  The  subject  was  also  expected  to 
recover  from  the  start  of  a  power-on  spiral  and  approach  to  a 
climbing  3tall  while  simulating  instrument  flight.  This 
simulation  was  graded  for  all  three  skill  categories. 

Change  in  Flight  Plan 

This  exercise  required  that  the  subject  proceed  to  an  al¬ 
ternate  destination  because  of  unexpected  conditions  which 
develop  in  flight  and  was  the  natural  follow-on  to  having  in¬ 
advertently  run  into  clouds  obstructing  the  route  to  the  ori¬ 
ginal  destination.  The  procedures  grade  reflected  the  action 
taken  to  recalculate  the  flight  planning  and  the  procedures 
used  to  advise  flight  service  of  the  change  in  destination. 

Radio  Procedures 

The  grade  for  procedural  skill  was  based  on  the  subjects' 
ability  to  determine  and  tune  the  correct  frequencies,  make  the 
required  calls  and  use  the  proper  phraseology.  The  judgment 
grade  was  included  because  there  is  so  often  a  problem  to  be 
solved  in  connection  with  radio  usage.  Typical  examples  are 
the  interpretation  of  tower  instructions,  answering  questions 
asked  by  the  tower  or  flight  service  and  knowing  how  to  get 
required  information  by  using  the  radio. 

Landings 

Grades  were  entered  on  every  observed  landing  and  the 
subject  was  always  expected  to  attempt  to  land  as  close  as 
possible  to  a  previously  designated  point  on  the  runway. 
Typically,  the  point  was  one  of  the  runway  centerline  marks. 
Grades  ranged  from  0-5  as  shown  cn  the  flight  Evaluation  Record 
Form  (Figure  3)  and  were  recorded  for  pattern  and  accuracy. 

The  grades  for  accuracy  were  determined  primarily  by  the  loca¬ 
tion  of  the  first  point  of  touchdown  relative  to  the  aim  point 
and  were  reduced  if  other  characteristics  of  the  landing  were 
poor.  There  was  also  a  grade  for  each  landing  pattern  listed 
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as  a  judgment  skill.  This  grade  reflected  the  entry  into  the 
pattern,  adherence  to  the  traffic  pattern  altitude,  maintenance 
of  correct  approach  speeds  and  safe  aircraft  attitude.  The 
first  and  last  flights  normally  included  5  or  6  landings  each. 
An  attempt  was  made  to  make  each  landing  different  by  asking 
for  different  flap  settings.  Comments  were  offered  on  the 
previous  pattern  and  landing  only  after  it  was  completed.  A 
grade  of  zero  was  awarded  if  the  approach  or  landing  was  so 
unsafe  that  the  Evaluator  had  to  take  over  control  of  the 
airplane. 

For  all  flights  grades  were  awarded  on  the  following: 
Aircraft  Preflight 
Start,  Taxi,  and  Run-Up 
Take-Off  and  Departure 
Simulated  Engine  Out 
Radio  Procedures 
Landings 
Overall  Grade 

For  the  first  and  last  flights  additional  grades  were  included 
on: 

Slow  Flight  and  Stalls 
Landings  (total  of  5  or  6) 

The  cross-country  flight  included  additional  grades  on: 

Flight  Planning  and  Filing 
VOR  Orientation  and  Tracking 
Simulated  Loss  of  Horizon 
Change  in  Flight  Plan 

Landings  (at  several  airports  if  feasible) 

The  subject  was  given  comments  on  his  performance,  but  no  in¬ 
formation  on  grades  or  the  grading  system. 

Subject  Selection 

Subjects  were  selected  for  the  flight  testing  program  from 
the  information  submitted  on  the  Pilots'  History  Questionnaire 
(Appendix  A-2) .  Selection  of  the  subjects  was  based  on  the 
distribution  of  the  Sample  Population  as  described  by  Figure  4. 
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SAMPLE  POPULATION 

RANGE 

RATIO 

Total  Flight  Time 

<150  :150--300  :300-600  :>600 

1:1:1:1 

Weeks  Since  Last  Flight 

<2:2-6:>6 

1:1:1 

Years  Since  Certification 

<3 : 3-6 : 6-15 :>15 

1:1:1 :1 

Type  Initial  Training 

FAA : Military : Other 

Approved  School 

2:1:1 

Age 

<30 : 30-37: 38-45 : >45 

1:1:1:1 

Gender 

Male  .'Female 

20:1 

Pilot  Category 

Civil  professional 

12:1 

Certificate  Held 

Private : Commercial 

5:1 

DISTRIBUTION 

BASED  ON  CURRENT  KNOWLEDGE 

OF  TOTAL  POPULATION 

i. 

Figure  4. 


A  breakdown  within  the  population  selected  is  shown  below: 

Sample  Population 
(1000  subjects) 

324  subjects  responded 
*28  not  eligible 

Flight  Testing  Selection  Process 
(296  subjects  eligible) 


Private  Pilots 
(270  eligible) 


Commercial  Pilots 
(26  eligible) 


Subjects  Selected 
60  male 
3  female 


Subjects  selected 
8  male 

0  female  (no  applications) 


Age  Groups 

Total 

Time  Range 

Years  Since  Certification 

Under  30 

29 

70-380 

hours 

(Range) 

1-7  years 

30-37 

19 

66-525 

hours 

1-12  years 

38-45 

13 

67-500 

hours 

1-15  years 

Over  45 

10 

75-5200 

hours 

2-15  years 

♦Eligibility  Criteria: 

1.  Private  or  Commercial  Pilot  Certificate 

2 .  Non-Instrument  Rated 

3.  Current  Medical  Certificate 

4.  Resides  within  50  miles  of  Hanscom  Field. 

Flight  Operations  Summary 

Of  the  seventy-one  subjects  selected  for  flight  testing, 
sixty-five  were  scheduled  and  completed  at  least  one  flight. 
Eight  subjects  subsequently  dropped  out  of  the  program  for 
personal  reasons  leaving  a  total  of  fifty  seven  who  completed 
the  three  flights.  The  data  for  two  of  these  subjects  was 
considered  incomplete  and  was  not  included  in  the  data  analysis. 
A  Flight  Operations  Summary  is  shown  in  Figure  5.  A  copy  of 
the  printed  introduction  to  the  program  (Appendix  D-l)  was  given 
to  the  subject  and  an  Experience  Data  Form  (Appendix  D-2}  was 
completed  prior  to  the  first  flight. 


FLIGHT  OPERATIONS  SUMMARY 

Flight  Flights  Flights  Cancelled  Flights  Flight 

No.  Scheduled  Weather-Other  Completed  Hours 

FI  110  Private 

16  Commercial 

126  45  16  65  83.4 

F2  116  Private 

40  Commercial 

156  73  24  59  120.7 


F3  74  Private 

25  Commercial 

99  30  12  57  79.6 


Totals  381 

1 


148  52  181  283.7 


Figure  5 


IV  ANALYSIS 


Data  Reduction 

The  various  measures  of  skill  were  analyzed  for  the  effects 
of  total  and  recent  flight  experience  by  computer,  using  a  set 
of  biostatistics  programs  (BMD  Statistics  Programs ,  Reference  4 
and  also  Appendix  F-l) .  Data  was  prepared  casewise  by  flight, 
and  punched  onto  80  column  cards.  (The  organization  of  these 
data  ^ards  is  given  in  the  Flight  Evaluation  Record  Coding 
Sheet,  shown  in  Appendix  E.)  Preparation  of  the  Flight  Eval¬ 
uation  Record  cards  from  the  individual  flight  evaluation  record 
forms  and  the  experience  data  involved  some  preprocessing  of  in¬ 
formation.  In  particular,  Total  Time,  Total  Time  in  Type 
(Cessna  150),  Recency,  Landing  Recency,  Type  Recency,  and  Type 
Landing  Recency  were  calculated  from  the  data  available  on  the 
flight  experience  forms  and  evaluation  records,  using  an  inter¬ 
active  FOCAL  language  program  shown  in  Appendix  F-2.  By  pre¬ 
processing  much  of  the  individual  pilot's  flight  history  in  this 
way,  the  total  amount  of  data  carried  into  the  final  analysis 
was  reduced,  and  the  data  structure  for  the  subsequent  BMD 
analysis  was  considerably  simplified.  Normally  the  Flight 
Experience  Data  form  was  filled  out  by  each  subject  just  prior 
to  the  first  evaluation  flight.  Care  was  taken  to  ascertain 
that  records  were  kept  of  all  subsequent  non-MIT  program  flights 
("in ter flights") ,  and  that  these  data  were  entered  into  the 
recency  calculations.  In  cases  were  there  was  doubt,  subjects 
were  contacted  by  mail  again  after  completing  the  program,  and 
asked  for  data  on  any  possible  interflights. 

Effect  of  Parametric  Experience  Variables  on  Overall  Grade 

In  order  to  assess  the  influence  and  relative  importance 
of  various  parametric  experience  variables  on  the  overall  grade 
received  by  a  subject  on  an  individual  flight,  a  stepwise  re¬ 
gression  analysis  was  performed  (using  the  program  BMD  02R) . 

The  program  attempts  to  find  a  linear  mathematical  relationship 
between  overall  grade  and  the  various  experience  factors  by 
fitting  the  data  with  a  multidimensional  surface  which  is 


optimal  in  a  least  mean  squares  sense.  The  stepwise  approach 
is  particularly  useful  in  experiments  such  as  this  where  the 
relative  order  of  importance  of  the  various  experience  factors 
is  not  known.  Basically,  the  program  operates  as  follows: 

It  selects  the  single  experience  variable  which  best  predicts 
overall  grade.  In  a  second  step,  the  program  finds  the  variable 
which  best  predicts  the  overall  grade  given  that  the  effect  of 
the  first  variable  on  overall  grade  has  been  removed.  In  the 
steps  that  follow,  either:  a)  a  variable  is  entered  which  best 
improves  the  prediction  of  skill  given  all  the  variables  entered 
from  the  previous  steps;  or  b)  a  variable  is  removed  from  the 
set  of  predictors  if  its  predictive  ability  falls  below  a  given 
level.  The  process  is  terminated  when  no  further  variable  im¬ 
proves  the  prediction  of  overall  grade. 

More  complete  descriptions  of  the  stepwise  regression  tech¬ 
nique  can  be  found  in  References  4  and  5.  Included  as  experience 
factors  were:  Age,  Years  since  Certification,  Total  Time,  Total 
Time  in  Type,  Overall  Recency,  Type  Recency,  Landing  Recency, 
and  Type  Landing  Recency.  The  program  was  run  successively 
using  several  different  transforms  (such  as  logarithmic  and 
exponential)  of  the  Total  Time  and  Recency  parameters  to  deter¬ 
mine  the  stepwise  regression  which  accounted  for  the  greatest 
amount  of  the  variance  in  the  overall  data.  The  resulting  re¬ 
gression  which  accounted  for  a  total  of  24.3%  of  the  variance 
in  the  overall  data,  wa3  based  of  the  following  model: 

Overall  Grade  =  KQ  +  (Cessna  150  Recency)  + 

K2  (Log^0 (total  time))  +  (years  certificated)  + 

(Log^Q (Cessna  150  time))  +  Kg (age)  +  EQ 

where  EQ  is  an  error  variable  representing  the  error  incurred 
in  approximating  skill  by  the  regression  function. 
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REGRESSION  COEFFICIENTS 


Partial 

Regression 

Coefficient 

Value 

Std.  Error 
of  Regression 
Coefficient 

%  of  Overall 
Variance  ac¬ 
counted  for 

K0 

1.42 

K1 

.25 

.09 

9.5 

X2 

.73 

.16 

7.4 

*3 

-  .030 

.01 

3.8 

.15 

.06 

2.5 

x5 

-  .0088 

.006 

1.1 

24.3% 

The  partial  regression  coefficient  represents  an  estimate 
of  the  effect  of  the  particular  associated  experience  variable 
on  overall  grade  when  the  other  experience  variables  are  held 
fixed.  Due  to  the  logarithmic  transformation  of  total  time 
variable,  some  of  the  coefficients  are  difficult  to  interpret 
directly.  However,  when  the  effects  of  the  logarithmic  trans- 
formations  are  taken  into  account,  one  can  observe  that  averaging 
across  the  sample,  pilots  who  have  either: 

1.  .4  hrs/week  more  type  recency,  or 

2.  32%  more  total  time,  or 

3.  3.3  fewer  years  since  certification,  or 

4.  150%  more  time  in  type 

have,  on  the  average,  a  0.1  skill  unit  higher  score. 

The  standard  error  of  the  regression  coefficient  gives  a 
measure  of  the  accuracy  to  which  a  particular  coefficient  is 
known.  In  particular,  it  is  an  estimate  of  the  standard  devia¬ 
tion  of  the  distribution  of  the  regression  coefficient  which 
would  be  obtained  if  the  same  analysis  were  performed  on  another 
set  of  similar  flight  data  from  the  same  population. 

The  individual  components  of  the  regression  shown  in  Table 
1  are  plotted  in  Figures  6,  7,  8,  9,  and  10.  These  figures  de¬ 
scribe  the  individual  effects  of  Type  Recency,  Total  Time, 

Years  Since  Certification,  Time  in  Type  and  Age,  on  overall  grade 
for  the  sample  population,  under  the  condition  that  all  inde- 
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pendent  variables  other  than  the  one  being  plotted  remain  con¬ 
stant.  Figure  11  shows  curves  of  constant  skill  as  a  function 
of  Type  Recency  and  Total  Time.  When  interpreting  these  figures, 
however,  it  is  important  to  keep  in  mind  that  the  regression 
model  is  merely  a  description  of  the  data  obtained  from  the 
sampled  pilots.  Although  the  regression  can  be  expected  to 
predict  the  mean  overall  grade  of  the  pilots  in  this  study,  it 
does  not  necessarily  prove  that  the  experience  varxables  are 
causal  in  any  sense,  however  attractive  this  hypothesis  may 
seem.  This  is  because  there  always  remains  the  possibility 
that  there  exist  parametric  experience  variables  other  than 
those  considered  in  this  study  which  exert  a  stronger  influence 
on  overall  grade,  and  which  happen  to  be  correlated  with  the 
independent  variables  found  by  the  regression  analysis. 

On  the  other  hand,  given  the  large  number  of  different 
experience  variables  considered  in  this  study,  it  is  difficult 
to  imagine  what  major  experience  factors  were  not  considered. 

From  the  results  of  this  study,  one  could  assert  that,  for 
example,  because  pilots  in  our  study  with  .4  hr s/week  more  Type 
Recency  had,  on  the  average,  0.1  skill  unit  higher  performance, 
it  seems  reasonable  that  a  pilot  who  increased  his  practise  rate 
by  that  amount  would  experience  a  corresponding  change  in  skill. 
Such  a  statement  cannot  be  regarded  as  "proven"  by  the  regression 
analysis  however.  This  difficulty  is,  of  course,  inherent  in 
the  process  of  model-making  using  statistical  techniques.  In 
defense  of  the  hypothesis  that  the  regression  parameters  may 
be  considered  causal,  it  can  be  said  that  none  of  the  signifi¬ 
cant  effects  found  by  the  regression  are  unexpected,  and  there¬ 
fore  that  there  exist  no  particular  grounds  for  rejecting  the 
hypothesis  that  the  regression  variables  are  causal. 

The  analysis  indicates  that  the  five  parametric  experience 
variables  can  account  for  a  total  of  24.3%  of  the  variance  in 
this  particular  set  of  data.  The  individual  partial  regression 
coefficients  are  statistically  nonzero.  The  probability  that 
they  are  zero  and  the  value  found  occured  by  chance  is  less 
than  .1%  for  log  of  total  time,  .5%  for  years  since  certifica- 
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tion  and  type  recency,  1%  for  log  of  total  time  in  type,  and 
15%  for  age. 

It  should  be  observed  that  stepwise  regressions  based  on 
untransformed  variables  or  including  transformations  of  other 
variables  or  with  certain  other  variables  witheld  could  account 
for  10-20%  of  the  overall  variance.  The  regression  given  pre¬ 
viously,  accounting  for  24.3%  of  the  variance,  was  the  most 
successful  attempted. 

The  finding  that  the  measured  experience  factors  account 
for  only  25%  of  the  overall  grade  indicates  that  other  factors 
and  interactions  contribute  significantly  to  pilot  performance. 
One  possibility,  previously  discussed,  is  that  there  exist  im¬ 
portant  experience  factors  common  to  all  pilots  other  than  the 
subset  considered  in  the  analysis  which  could  account  for  an 
additional  component  of  the  variance.  More  probable  sources 
of  the  remaining  variance  are  the  three  non-experience  related 
groups  of  factors  discussed  in  Section  II  (Theory) . 

Operational  constraints  in  test  program  during  the  winter 
months  prevented  the  use  of  a  balanced  design  in  which  each 
subject  flew  with  each  Jf  only  three  instructors.  Hence,  it 
was  not  possible  to  directly  determine  the  component  of  overall 
variance  due  to  instructors.  It  was  observed  that  although  all 
instructors  gave  a  median  grade  of  3,  the  instructors  were 
found  to  differ  in  a  statistically  distinguishable  way.  The 
effect  of  these  differences,  and  "roundoff"  error  associated 
with  the  use  of  integer  grades  undoubtedly  contributes  to  the 
overall  variance. 

It  was,  however,  possible  to  assess  the  component  of 
variance  due  to  fixed  (latent  skill)  effects  associated  with 
individual  subjects.  (Group  1  in  Section  II).  The  regression 
model  previously  described  predicts  the  average  skill  of  the 
sample  population.  The  overall  skill  of  an  individual  pilot 
may  be  considered  as  predicted  by  the  regression  model  plus  an 
additional  constant  term  due  to  his  own  latent  ability. 

Looking  across  the  sample  population,  the  additional  constant 
term  follows  a  distribution  which  has,  necessarily,  zero  mean. 
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and  an  estimated  standard  deviation  of  .72.  This  was  determined 
by  an  analysis  of  variance  of  the  overall  grades  with  the 
effects  accounted  for  by  the  regression  removed.  (Examination 
of  regression  residuals  in  this  way  effectively  constitutes  an 
analysis  of  covariance.) 

F'xed  effects  due  to  the  55  subjects  in  165  flights  was 
found  to  contribute  303  of  the  overall  variance  in  all  the  data. 
Thus,  the  regression  model  taken  together  with  a  consideration 
of  latent  skill  effects  can  account  for  over  half  the  variance 
in  the  overall  skill  grades.  Sources  of  the  remaining  45%  of 
the  variance  in  the  data  include  such  factors  as:  effects  due 
to  instructors,  "roundoff"  error,  and  the  individual  pilot's 
flight  to  flight  variability  in  performance. 

Individual  Skill  Grades 

A  statistical  summary  of  the  parametric  flight  data  vari¬ 
ables  is  given  in  Table  2.  This  table  was  generated  by  the 
program  BMD  01D.  The  mean  is  the  average  of  the  variables  over 
all  flights  for  which  data  was  taken.  The  standard  error  of 
the  mean  is  the  standard  deviation  divided  by  the  square  root 
of  the  sample  size.  It  estimates  the  standard  deviation  of  the 
distribution  of  means  one  might  obtain  if  the  study  was  to  be 
repeated  many  times.  Thus,  it  is  indicative  of  the  confidence 
which  can  be  placed  in  the  listed  means.  The  sample  size  is 
the  number  of  flights  on  which  the  variable  was  recorded.  The 
maximum  value,  minimum  value,  and  range  are  self-explanatory. 

Several  observations  can  be  made  from  this  table: 

The  Mean  Age,  Total  Time,  and  Years  Certificated  compare 
well  with  the  values  known  for  the  total  population. 

The  Landing  Recency  divided  by  the  mean  Recency  (both  over¬ 
all  and  for  Type)  gives  an  approximate  landing  rate  of  two 
landings  per  flight  hour. 

The  average  quiz  score  was  7/10. 

An  interesting  observation  can  be  made  .by  considering  the 
individual  skill  grades  (landings  excluded).  The  four  highest 
and  four  lowest  scores  are  shown  in  Figure  12.  Of  interest  is 
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Retention  and  Recall  P  =  Judgment  and  Problem  Solving  M  =  Motor  Coordination 
STATISTICAL  SUMMARY  OF  PARAMETRIC  VARIABLES 


the  observation  that  the  poorest  average  scores  are  on  stalls 
and  flight  by  instruments.  The  Evaluator  reported  that  many 
of  the  subjects  had  not  done  stalls  since  their  initial  flight 
training.  Some  subjects  had  no  simulated  instrument  experience 
at  all.  On  the  other  hand,  the  above  average  skills  appear  to 
be  ones  which  are  practiced  fairly  often. 

As  a  check  on  the  hypothesis  that  the  overall  grade  was  a 
"mental"  average  by  the  Evaluator  of  the  subjects'  individual 
skill  scores,  the  mean  of  all  these  scores  taken  together  was 
calculated.  Its  value  was  3.05  compared  to  2.86  for  the  actual 
overall  grade  mean.  Similar  means  were  calculated  for  skill 
categories  defined  as  recall,  problem  solving  ability,  and  motor 
skills.  The  means  here  were  3.06,  3.07,  and  3.02  respectively. 


Correlations 

A  number  of  the  individual  skill  scores  were  also  found  to 
be  correlated  with  the  overall  grade  using  the  program  BMD  03D. 
The  correlation  coefficient,  p,  is  an  indicator  of  the  strength 
of  linear  association  of  two  variables.  The  correlation  coeffi¬ 
cient,  p  is  defined  by 

p  A  sTsT 


where 


xi> (xjk 


This  equation  represents  the  sum  of  the  product  of  the  deviation 
of  X^  and  X..  from  their  respective  means.  By  definition  this 
is  the  covariance  of  X^  and  Xj. 


This  equation  represents  the  sum  of  the  squares  of  the  devia¬ 
tions  of  X^  from  its  mean.  This  by  definition  is  the  variance 
of  X^  or  the  square  of  the  standard  deviation  of  X^.  The  corre¬ 
lation  coefficient  is  thus  the  ratio  of  the  covariance  to  the 
product  of  the  two  standard  deviations.  By  definition  |  p  |  <^1 , 

.  with  p  =  +  1  representing  an  exact  linear  dependence  of  the  two 


variables  (i.e.,  variables  and  X2  could  be  explained  exactly 
by  an  equation  of  the  form  X2  ■  mX^  +  b  where  m  and  b  are  con¬ 
stants)  .  A  positive  sign  for  p  indicates  a  change  in  one  of 
the  variables  results  in  a  corresponding  change  (i.e.,  both  in¬ 
crease  or  decrease  simultaneously)  in  the  other.  On  the  other 
hand,  a  negative  sign  for  p  indicates  the  variables  change 
simultaneously  but  in  opposite  directions.  The  closer  p  is  to 
1  the  greater  the  linear  association  between  the  two  variables. 

The  variables  for  which  the  correlation  coefficient  indi¬ 
cated  some  linear  dependence  are  given  below.  Overall  grade 
correlated  with : 

Flight  Planning,  p  *  .7 
Stall  (motor  skills),  p  *  .6 
Take-Off,  p  =  .6 
VOR,  p  =  .6 

Pilotage  and  Dead  Reckoning,  p  »  .6 
Change  in  Flight  Plan,  p  =  .6 
Radio  Skills,  p  =  .6 

These  correlations  suggest  that  overall  grade  varies  in  a 
linear  fashion  with  the  individual  skill  scores  indicated, 
further  supporting  the  hypothesis  that  the  overall  grade  is  a 
composite  of  these  scores. 

Other  correlations  of  interest  were: 

Overall  Recency  and  Landing  Recency,  p  =  .89 
Type  Recency  and  Type  Total  Time,  p  =  .7 
Type  Recency  and  Type  Landing  Recency,  p  =  .9 

The  correlations  between  Recency  and  Landing  Recency  both 
overall  and  in  Type  agree  with  intuition.  The  correlation  be¬ 
tween  Type  Recency  and  Type  Total  Time  may  indicate  that  pilots 
fly  often  in  the  same  type  aircraft,  a  not  unreasonable  assump¬ 
tion. 

It  must  be  pointed  out  that  the  correlation  coefficient  by 
itself  may  not  always  be  simple  to  interpret,  especially  when 
it  is  below  0.7.  Above  this  value  however,  on  can  expect  to 
see  a  reasonably  linear  relationship  between  the  two  variables 
if  they  are  plotted.  An  example  of  such  a  plot  for  Overall 


Recency  vs.  Overall  Landing  Recency  is  shown  in  Figure  13.  As 
mentioned  above,  the  correlation  coefficient  here  is  approxima¬ 
tely  0.9  and  the  plot  is  indeed  nearly  a  straight  line. 

Examination  of  Non-Paramatric  Data 

Correlation  coefficients  can  not  be  defined  on  the  non- 
parametric  data  in  the  study  such  as  Evaluators,  type  of  ope¬ 
ration,  and  airfield  type.  Relationships  between  these  param¬ 
eters  and  certain  of  the  parametric  variables  were  examined 
using  the  Chi-squared  analysis  of  contingency  tables  technique 
(Reference  5,  p.  77ff ) .  A  computer  program,  CONTAB,  wan  written 
in  the  interactive  language,  FOCAL,  to  perform  such  analyses. 

A  copy  of  this  program  is  included  in  Appendix  F-3. 

Those  relationships  of  interest  in  which  some  dependence 
of  parameters  was  found  are  as  follows : 

Airfield  Type  and  Radio  Skills,  p  <  0.001  (the  probability , 
p,  that  this  dependence  occurs  by  chance  is  <  0.1%) 

Radio  scores  were  higher  for  pilots  regularly  flying 
from  controlled  airfields. 

Airfield  Type  and  Overall  Grades,  p  <  0.05 

Overall  grade  was  higher  for  pilots  regularly  flying 
from  controlled  airfields. 

Relationships  of  some  interest  for  which  no  dependence  of 
parameters  could  be  demonstrated  were: 

Training  Type  and  Overall  Grade 
Operation  Type  and  Overall  Grade 
Airfield  Type  and  Landing  Skills 
Overall  Grade  and  Written  Quiz  Score 

The  mean  landing  pattern  scores  and  the  mean  landing 
accuracy  scores  are  plotted  in  Figures  14  and  15.  It  is  in¬ 
teresting  to  note  the  slow  but  steady  improvement.  Note  that 
the  increase  is  small  relative  to  the  magnitude  of  the  sample 

cf  anrlarH  dSVi.2tti.OIi*  ThS2TS  2pp92TS  tO*  £111  CffCCt  CIV  the 

average  although  it  is  difficult  to  observe  the  effect  when 
looking  at  only  a  few  landings  of  a  single  pilot.  However,  it 
would  not  be  correct  to  assume  that  actual  overall  landing  skill 
in  a  non-test  situation  increases  in  exactly  this  way  with 
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practice/  because  in  this  study,  repeated  landings  were  normally 
made  on  the  same  runway.  It  is  reasonable  to  expect  that  the 
pilot  adjusts  to  that  particular  landing  pattern  and  touch¬ 
down  point.  The  landing  skill  scores  would  probably  not  in¬ 
crease  as  rapidly  if  landings  were  made  consecutively  to  dif¬ 
ferent  airports. 
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V 


CONCLUSIONS 


This  work  has  been  a  study  of  the  observed  skills  of  a 
sample  of  non-ins trumont  rated,  private  and  commercial  pilots. 
The  test  sample  consisted  of  55  subjects  who  flew  three  flights 
with  an  evaluator  in  a  Cessna  150.  The  sample  appeared  repre¬ 
sentative  of  the  population  in  terms  of  such  factors  as  age  and 
experience.  Several  experience  factors  were  found  which  showed 
correlation  with  the  observed  piloting  skill  of  the  sample. 

In  order  to  make  these  correlations  it  has  been  necessary  to 
look  at  the  skills  of  the  sample  population  as  a  whole.  It  was 
not  possible  to  perturb  the  experience  factors  of  an  individual 
pilot  enough  to  see  the  effect  on  his  individual  overall  skill. 
Consequently,  it  has  not  been  proven  that  one  can  cause  individ¬ 
ual  skill  changes  by  changing  experience  factors.  That  is  an 
assumption  which  is  encouraged  by  the  fact  that  the  population 
shows  these  characteristics.  In  explaining  the  conclusions  it 
is  implied  that  these  statistics  about  the  population  can  be 
applied  to  an  individual.  The  reader  should  be  aware  that  the 
study  only  described  the  characteristics  of  the  sample  popula¬ 
tion  and  that  interpretations  on  individuals  requires  a  postu¬ 
late  which  was  not  proven.  What  is  meant  in  talking  about 
"a  pilot"  means  what  would  happen  on  the  average  to  the  total 
population  under  the  assumption  that  experience  changes  would 
cause  skill  changes,  in  the  same  way  in  which  they  varied  over 
the  sample.  Keeping  this  subtle  distinction  in  mind,  the  fol¬ 
lowing  major  conclusions  are  drawn  from  the  results: 

1.  Experience  factors  account  for  only  about  25%  of  the 
variance  in  observed  skill.  Effects  due  to  the  latent  skill 
of  each  individual  subject  account  for  about  30%.  It  is  there¬ 
fore  probably  necessary  to  fly  with  an  individual  pilot  in  order 
to  accurately  measure  his  skill.  This  large  variation  between 
individuals  makes  prediction  of  an  individual  pilot's  skill 
based  only  on  experience  factors  unreliable.  However,  effects 
due  to  skill  may  be  observed  when  looking  across  a  population 
of  pilots. 
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2.  The  most  important  experience  factor  for  maintaining 
skill  is  recent  flight  experience  as  measured  by  Recency  which 
is  approximately  the  average  rate  at  which  the  skill  is  prac¬ 
ticed.  Recency  predicts  the  largest  percentage  of  the  variance 
in  skill  grades  due  to  experience  factors  and  is  the  single 
experience  factor  over  which  the  pilot  has  immediate  control. 

3*  pother  important  factor  showing  correlation  in  a 
regression  model  for  skill  involved  the  logarithm  of  total  time. 
The  log  dependence  comes  from  the  apparent  fact  that  skill  in¬ 
creases  with  total  time  at  a  rate  that  is  inversely  proportional 
to  the  total  time  accumulated. 

4.  Prom  a  statistical  viewpoint ,  the  increase  of  skill 
with  the  log  of  total  time  is  offset  by  a  linear  decrease  of 
skill  with  years  since  certification.  In  the  regression  model 
after  a  large  number  of  total  hours  builds  up,  the  linear  de¬ 
crease  of  skill  due  to  years  since  certification  overpowers 
the  logarithmic  increase  due  to  total  time.  The  hypothetical 
effect  is  shown  graphically  in  Figure  16  for  a  pilot  who  con¬ 
sistently  flys  100  hours  a  year.  If  the  effect  due  to  years 
since  certification  is  causal,  the  trend  could  be  countered  by 
qualifying  for  a  new  certificate  or  flying  at  a  greater  rate. 
However,  this  interpretation  should  only  be  accepted  cautiously. 
The  effect  can  also  be  attributed  exclusively  to  the  fact  that 
the  regression  only  describes  pilots  from  the  sample  population 
without  advanced  ratings.  Pilots  still  in  this  population  after 
many  years  may  not  be  typical  of  the  overall  population  of  all 
pilots.  More  probably  both  effects  are  at  work  simulatenously 

5.  There  was  no  significant  relationship  between  the  score 
on  written  quizzes  and  overall  grade.  Neither  did  the  subject's 
own  assessment  of  his  own  skill  show  significant  relationship 
with  overall  grade.  There  was  no  significant  relationship  found 
between  skill  and  geographic  factors.  There  was  no  significant 
relationship  between  skill  and  type  of  initial  training. 

6.  Individual  skills  receiving  the  highest  grades  on  the 
average  were  those  probably  most  often  practiced  such  as  flight 
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planning,  preflight,  taxi  and  take  off.  Those  skills  receiving 
the  lowest  average  grades  involved  stalls  and  simulated  instru¬ 
ments  flight.  Most  of  the  subjects  had  practiced  these  skills 
little  or  never  since  their  initial  training.  It  is  interesting 
that  inadvertent  stalls  and  flight  into  instrument  weather 
contribute  to  a  major  percentage  of  the  accidents  to  pilots  in 
the  population  according  '*o  NTSB  accident  statistics. 
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MASSACHUSETTS  INSTITUTE  OP  TECHNOLOGY 
DEPARTMENT  OP  AERONAUTICS  AND  ASTRONAUTICS 
Measurement  Systems  Laboratory 
Bldg.  N52-395 

Cambridge,  Massachusetts  02139 


Dear  Sir: 

The  Massachusetts  Institute  of  Technology  is  conducting  an  in¬ 
vestigation  to  learn  more  about  the  flying  skills  of  private 
and  commercial  pilots ‘in  the  U.S.,  how  these  skills  vary  with 
recent  and  total  pilot  experience  and  the  extent  to  which  pilot 
skills  degrade  when  not  exercised. 

If  your  response  to  the  enclosed  questionnaire  indicates  that 
your  total  flying  experience  is  in  some  way  typical  of  the 
pilot  population,  (regardless  of  recent  flight  experience)  you 
are  welcome  to  participate  in  this  research  program. 

If  you  are  selected  as  a  candidate  for  this  research  program, 
you  will  be  given  -at  no  cost  to  you-  at  least  two  flights  and 
possibly  as  many  as  five  flights  with  a  Certified  Flight  In¬ 
structor  at  an  airfield  near  your  home.  You  will  do  all  the 
flying  and  will  be  asked  to  perform  only  those  maneuvers  with 
which  you  are  familiar.  Sufficient  latitude  in  time  will  be 
available  during  these  flights  to  permit  instruction  where 
appropriate. 

We  think  you  will  enjoy  this  experience  and  we  hope  that  we  can 
register  you  as  a  volunteer  to  receive  this  free  flight  time. 

Xf  you  are  interested  in  participating,  please  complete  and 
return  the  questionnaire  as  soon  as  possible.  Should  you  choose 
no*-  to  participate,  by  filling  out  and  returning  the  question¬ 
naire  you  can  help  assure  the  statistical  validity  of  the  in¬ 
vestigation. 


Respectfully, 


Walter  M.  Hollister. 


WMH/m 

Enclosure 


Appendix  A-l 
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PILOTS  HISTORY  QUESTIONNAIRE 

NAME _ DATE  OP  BIRTH _ 

Last  First  Middle  Mo.  Day  Year 

Address 


Street  City  State  Zip 

Home  Telephone _  Business  Telephone _ 


1.  Pilot  Certificate: 

Commercial a 

Private  □ 

Ratings  and  Limitations : 

Issue  Date 

Month 

Year 

2.  Medical  Certificate: 

Class : 

Its  is  current: 

3.  Source  of  Initial  Flight  Training  (Military, 
School,  etc.) 

Certificate  Source  Model  Aircraft 

Commercial 

Civilian  Flight 

Dates  Attended 

Private 

Remarks :  List  other  flight  training  completed  and  pilot  quali¬ 
fications  not  included  in  current  FAA  certification  and  ratings: 

List  all  single  engine  aircraft  models  in  which 
fied: 

you  have  quali- 

4.  Flight  Time  Summary: 

a.  Past  3  Months 

Model  Aircraft _ _ 

Flight  Time  (Hr.) _ _  _ 

Cross  Country  (Hr.) _ _ 

Night  Time  (Hrs.) _ 

No.  of  Landings  _ 

b.  Total  Flight  Time  During  Past  Year _ 

c.  Total  All  Flight  Time  _ 

5.  Have  you  flown  regularly  for  hire?  _ 

6.  Do  you  keep  a  current  pilots  Log  Book?  _ 

7.  How  do  you  evaluate  your  present  pilot  skill  level  relative 
to  your  skill  at  the  time  you  were  last  certificated? 

Same _  Better  _  Not  so  good _ 


Appendix  A-2 


37 


8.  Assume  that  you  were  flying  regularly  at  the  rate  of  1  hour 

per  week  and  that  you  stopped  all  flying  for  a  period  of  6 
months.  How  many  flight  hours  would  you  estimate  you  would 
need  to  regain  your  former  pilot  proficiency?  _ Hours. 

9.  Can  you  participate  in  the  flight  test  portion  of  this 

program? _ 

10.  If  selected  as  a  candidate,  on  what  days  would  you  be 
available  for  flight  testing? 

Mon. _  Tues.  _ Wend.  _ Thurs.  _  Fri.  _ 

Sat.  _  Sun.  _ _ 

I  am  not  available  during  the  following  period: _ 
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MATHEMATICAL  DEFINITION  OF  RECENCY 


Recency  is  defined  as  the  current  rate  at  which  a  skill 
is  being  practiced.  The  problem  is  that  the  value  of  the 
current  rate  has  to  be  averaged  over  a  suitable  time  interval. 
This  averaging  interval  is  intimately  associated  with  the  de¬ 
gradation  of  skill  with  time  under  two  simple  assumptions: 

1)  the  Recency  increases  at  a  rate  in  direct  proportion  to  the 
time  spent  practicing;  and  2)  the  percentage  decrease  in  Re¬ 
cency  with  time  is  proportional  to  the  time  spent  without  prac¬ 
tice. 

S  *  demonstrable  skill  at  time  t  as  scored  by  a  competent 
evaluator 

H  =  total  number  of  hours  (or  repetition)  of  practice  of 
the  skill 

R  =  Recency 

After  practice 

AR  =  AAH  (1) 

and  without  practice 

—  -  -  BAt  (2) 

With  regular  practice  under  steady  state  conditions  the 
two  increments,  AR,  must  be  equal.  By  definition,  the  value  of 
steady  state  P  is  AH/At.  Then  the  two  constants  of  proportion¬ 
ality  (A  and  E)  are  equal,  and  they  have  the  units  of  reciprocal 
time .  Now  let 

T  =  characteristic  degradation  time  and  set 


Equations  (1)  and  (2)  may  be  combined  to  form  an  expression 
for  the  total  AR,  and  this  expression  can  be  passed  to  the  limit 
as  At  approaches  zero  to  obtain 

tR  +  R  »  H  (3) 

Equation  (3)  or  the  approximations  (1)  and  (2)  can  be  used 
to  determine  the  current  value  of  Recency  for  any  pilot  once  r 
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is  known.  Conversely  a  characteristic  skill  degradation  time 
for  the  pilot  can  be  obtained  by  observing  how  his  performance 
improves  or  degrades  with  or  without  practice.  The  variation 
of  skill  with  Recency  is  a  function  that  might  be  character¬ 
ized  as  in  Figure  17. 


Figure  17.  Variation  of  Skill  with  Practice  Rate  (Recency) . 

This  figure  shows  ST  as  the  latent  skill  level  without 
benefit  of  practice,  SQ  as  a  minimum  acceptable  skill  level, 
and  RQ  as  the  corresponding  Recency  required  to  maintain  that 
minimum  skill  level.  For  any  pilot  the  change  in  S  from  SQ 
to  can  be  expressed  through  a  Taylor  Series  Expansion  around 
the  point  (RQ,  Sq)  as 

-  SQ  =  K(R^  -  Rq)  +  higher  order  terms  (4) 

Under  the  assumption  that  the  rate  of  accumulation  of  flying 
hours  for  the  pilot  is  available  from  his  log  book  and  given 
three  measures  of  skill  at  three  different  times,  it  is 
theoretically  possible  to  determine  for  every  pilot  that  is 
evaluated: 

a)  t,  Recency  time  constant  characteristic  degradation 
time; 
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b)  K,  the  rate  of  change  of  skill  with  respect  t 

c)  Rq,  the  minimum  Recency  required  to  maintain 
acceptable  skill  level. 


Recency 

minimum 
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Quiz-Plight  #1 


1)  An  aircraft  may  stall: 

(a)  only  when  in  a  nose-high  attitude  to  the  horizon. 

(b)  only  when  at  low  airspeeds. 

(c)  at  any  airspeed  and  in  any  attitude. 

(d)  only  when  both  (a;  and  (b)  are  true. 

2)  Previous  clearance  must  be  received  from  using  or  con¬ 
trolling  agency  before  flying  into: 

(a)  a  Warning,  Restricted  of  Prohibited  Zone. 

(b)  only  a  Restricted  or  Prohibited  Zone. 

(c)  only  a  Prohibited  Zone. 

(d)  only  a  Restricted  Zone. 

3)  According  to  the  F.A.R. 's,  when  flying  a  magnetic  course 
between  0°  and  179°  inclusive  at  3,000  feet  or  more  above 
the  surface,  the  aircraft  shall  be  flown  at: 

(a)  even  thousands  feet  altitude  aboven  the  surface. 

(b)  odd  thousands  feet  altitude  above  the  surface 

(c)  even  thousands  +  500  feet  altitude  above  the  surface. 

(d)  odd  thousands  +  500  feet  altitude  above  the  surface. 

4)  Carburetor  ice  forms: 

(a)  only  when  the  air  temperature  is  below  32°F  and  the 
humidity  is  high. 

(b)  especially  readily  in  a  20°-70c'F  temperature  range  with 
high  humidity. 

(c)  only  when  there  is  condensation  in  the  air  3uch  as  fog, 
rain,  snow  or  sleet. 

(d)  only  when  the  engine  is  throttled  back  and  the  humidity 
is  high. 

5)  A  pilot  has  his  VOR  omni  bearing  selector  set  on  280°,  his 
course  deviation  needle  centered  and  his  to/from  indicator 
reading  "to"  for  a  certain  VOR  station.  From  this  infor- 


Appendix  C-l 


42 


mation  one  can  assume  that: 

(a)  he  is  on  the  280°  radial  and  has  a  magnetic  heading 
of  280°. 

(b)  he  is  on  the  100°  radial. 

(c)  he  is  on  the  100°  radial  with  a  magnetic  heading  of 
280°. 

(d)  he  is  on  the  280°  radial  with  a  magnetic  heading  of 

100°. 

6)  The  effective  power  output  of  an  aircraft  reciprocating 
engine  will: 

(a)  decrease  with  higher  temperature  and  humidity. 

(b)  increase  with  higher  temperature  and  humidity. 

(c)  remain  the  same  for  all  temperatures  and  humidities. 

(d)  increase  with  higher  humidity  and  decrease  with  higher 
temperature. 

7)  The  VFR  weather  minimums  in  a  control  zone  are  500'  verti¬ 
cally  under,  1,000'  vertically  over,  2,000'  horizontally 
from  clouds  and: 

(a)  a  ceiling  of  1,000  feet  and  3  miles  visibility. 

(b)  a  ceiling  of  1,000  feet  and  1  mile  visibility. 

(c)  a  ceiling  of  2,000  feet  and  3  miles  visibility. 

(d)  a  ceiling  of  2,000  feet  and  1  mile  visibility. 

8)  The  special  VFR  minimums  for  a  control  zone  which  go  into 
effect  after  an  appropriate  ATC  clearance  is  received  are: 

(a)  3  miles  visibility,  1.000  feet  ceiling. 

(b)  1  mile  visibility,  clear  of  clouds. 

(c)  3  miles  visibility,  clear  of  clouds. 

(d)  1  mile  visibility,  1,000  feet  ceiling. 

9)  If  two  aircraft  are  approaching  each  other  at  approxi¬ 
mately  the  same  altitude  on  an  apparent  collision  course, 
one  aircraft  gives  way  to  the  other  if  the  other  aircraft: 
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(a)  is  larger. 

(b)  is  slower. 

(c)  is  approaching  from  the  left. 

(d)  is  approaching  from  the  right. 

10)  Over  congested  areas  the  minimum  clearance  of  the  highest 
obstacles  must  be: 

(a)  500  feet  above,  2,000  feet  horizontal. 

(b)  500  feet  above,  1,000  feet  horizontal. 

/. c)  1,000  feet  above,  2,000  feet  horizontal. 

(d)  1,500  feet  above,  2,000  feet  horizontal. 
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Quiz-Flight  #2 


1)  The  maximum  terrain  elevation  numbers  on  a  sectional  chart 
indicate : 

(a)  the  highest  terrain  in  the  area. 

(b)  the  highest  terrain  and/or  man  made  obstructions  in 
the  area. 

(c)  the  highest  man  made  obstructions  only  when  they  are 
higher  than  the  terrain  in  the  area. 

(d)  none  of  the  above. 

2)  All  radio  navigation  aids  on  sectional  charts  are  laid  out 
in  relation  to: 

(a)  the  true  north  line. 

(b)  the  magnetic  north  line. 

(c)  the  longitude  lines. 

(d)  none  of  the  above. 

j)  On  a  sectional  chart  the  shadeu-in  airport  symbol  denotes: 

(a)  airports  w:  oh  a  control  tower  or  a  FSS. 

(b)  airports  with  service  facilities. 

(c)  airports  with  hard-surfaced  runways  of  at  least  1500'. 

(d)  restricted  airports. 

4)  Magnetic  heading  plus  or  minus  Deviation  equals: 

(a)  compass  heading. 

(b)  magnetic  course. 

(c)  true  heading. 

(d)  true  course. 

5)  Winds  aloft  forecasts  are  given  in: 

(a)  kilometers  per  hour. 

(b)  miles  per  hour. 

(c)  knots  per  hour. 

(d)  knots. 
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6)  With  regard  to  completion  of  flight  plans,  it  is  true  that 

(a)  At  airports  with  control  towers  the  tower  will 
automatically  close  flight  plans  only  if  it  is  the 
airport  of  destination  specified  in  the  flight  plan. 

(b)  The  arrival  notice  must  be  filed  by  the  pilot  within 
an  hour  after  arrival  on  a  standard  form  provided  for 
the  purpose. 

(c)  The  pilot  should  request  the  FSS  to  close  the  flight 
plan. 

(d)  At  all  airports  with  a  control  tower  the  tower  auto¬ 
matically  will  close  flight  plans  as  soon  as  the 
landing  is  complete. 

7)  If  an  aircraft  enroute  on  a  cross-country  flight  passes 

within  five  miles  of  a  tcwer-controlled  airport  its 

minimum  altitude  should  bz: 

(a)  2,000  feet  ASL. 

(b)  2,000  feet  AGL. 

(c)  3,000  feet  ASL. 

(d)  3,000  feet  AGL. 

8)  On  aircraft  equipped  with  fixed-pitch  propellers  the  indi¬ 
cation  of  carburetor  icing  would  lively  be: 

(a)  a  decrease  in  engine  RPM. 

(b)  engine  roughness  only. 

(c)  a  loss  of  power  only. 

(d)  any  of  the  above. 

9)  The  scale  of  sectional  aeronautical  charts  is: 

(a)  1:250,000. 

(b)  1:500,000. 

(c)  1:1,000,000. 

(d)  1:1,500,000. 
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10)  The  most  important  reason  for  servicing  fuel  tanks  to  full 
capacity  upon  completion  of  a  flight  is  because  this  pro¬ 
cedure: 

(a)  prevents  drying  and  cracking  of  the  fuel  cell  inner 
liner  which  occurs  when  it  is  exposed  to  the  air. 

(b)  minimizes  the  possibility  of  corrosion  and  structural 

f  damage  due  to  moisture  forming  on  and  dripping  from 

the  outer  walls  of  fuel  tanks. 

(c)  prevents  the  fuel  evaporation  which  occurs  in  partially 
fillled  tanks . 

(d)  minimizes  the  possibility  of  fuel  contamination  from 
condensation  of  water  on  inner  walls  of  partially- 
filled  tanks. 

i 

L 


< 
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INTRODUCTION  TO  THE  PILOTING  SKILLS  RESEARCH  PROGRAM 


The  purpose  of  this  program  is  to  obtain  data  which  will 
indicate  how  experience  factors  influence  observed  piloting 
skill. 

The  scope  of  the  program  involves  flights  with  approxi¬ 
mately  100  private  and  commercial  pilots  who  are  neither  in¬ 
strument  nor  multi-engine  rated.  The  plan  is  to  obtain  data 
on  experience  factors  and  observe  the  flying  skills  of  each 
subject.  These  data  will  be  processed  by  a  computer  program 
which  will  look  for  statistical  correlations  between  observed 
skills  and  experience  factors. 

A  typical  program  consists  of  the  following  steps: 

1.  Take  data  on  your  experience  with  the  aid  of  your  log 
book. 

2.  Ten  question  quiz. 

3.  Basic  piloting  flight. 

4.  Cross-country  flight. 

5.  Idle  period  of  3-4  weeks. 

6.  A  repeat  of  the  first  flight. 

During  the  3-4  week  idle  period,  it  would  be  appreciated 
if  you  did  not  fly.  If  you  should  fly  during  that  period,  we 
would  like  the  information  on  flight  time,  landings,  etc.  from 
your  log  book. 

During  the  flights,  you  should  do  things  the  way  you  feel 
they  should  be  done.  The  experimenter  will  be  pilot  in  command, 
but  will  act  as  observer  pilot.  At  the  end  of  the  flight,  he 
will  comment  on  his  observations.  However,  his  remarks  and 
your  test  results  will  be  completely  confidential:  You  as  a 
subject  are  identified  in  the  data  only  as  a  number.  Our  only 
record  of  names  vs.  numbers  will  be  destroyed  after  the  project 
is  completed.  The  number  code  used  will  be  changed  during  the 
processing  of  the  data  so  that  no  one  (including  yourself)  will 
be  able  to  associate  the  data  with  a  particular  individual. 

The  aircraft  you  will.be  flying  is  a  late  model  Cessna  150, 
maintained  on  a  100  hour  inspection  schedule.  The  aircraft  is 
covered  by  property  damage  and  liability  insurance. 
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You  are,  of  course,  completely  free  to  discontinue  your 
participation  in  this  study  at  any  time,  should  you  desire  to 
do  so. 

If  you  have,  at  any  time,  further  questions  about  the 
program,  please  do  net  hesitate  to  ask. 


Subjects:  (Please  sign) 

I  hereby  consent  to  take  part  in  the  piloting  skills 
research  program. 
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EXPERIENCE  DATA 


SUBJECT  NO. _ 

DATE  PRIVATE _ 

MO  YR 

TYPE  INITIAL  TRAINING 

_ FAA  APPROVED  SCHOOL 

_ MILITARY 

_ OTHER 

LOCAL  TERRAIN 

_ MTS 

_ FLAT 

_ COASTAL 

MAJORITY  OF  OPERATIONS 


DATE  OF  BIRTH _ 

SEX  _ M  _ F 

DATE  COMMERCIAL  _ 

MS  YR 

PROFESSIONAL  PILOT 

_  YES 

_  NO 

AIR  TRAFFIC  DENSITY 

_ HIGH  DENSITY 

LOW  DENSITY 


TOWER  CONTROLLED  FIELD 
UNCONTROLLED  FIELD 


LAST  10  FLIGHTS  IN  1972 


Julian  Date 

Flight  Duration  (HRS.) 

No.  Landings 

if  C-150 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
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EXPERIENCE  DATA  CONT 


SUBJECT  NO. 

DATA  JUST  PRIOR  TO  LAST  FLIGHT  SHOWN  ABOVE: 


TOTAL  TIME  (HRS . ) _ 
TOTAL  LANDINGS  (NO) 
C-150  TIME  (HRS)  _ 
C-150  LANDINGS  (NO) 


RECENCY  (HRS/WK)  _ 

LANDING  RECENCY  (NO./WK) 

C-150  RECENCY  (HRS/WK)  __ 

C-150  LANDING  RECENCY 

(NO/WK)  _ 


OWN  ESTIMATE  OF  RECENCY  TIME  CONSTANT 
(WKS) _ * _ 

OWN  ESTIMATE  OF  SKILL  RELATIVE  TO  WHEN  CERTIFICATED: 
IMPROVED 


SAME _ 

DETERIORATED 
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FLIGHT  EVALUATION  RECORD  CODING  SHEET 


Card  Column 

Item 

1-3 

Subject  Identification  Number 

From  Experience  Data 

Form:  7  or  77  missing  data 

4 

Pilot  Certificate: 

1  =  Private 

3  =  Commercial 

5-6 

Year  of  Issuance 

7-8 

Month  of  Issuance 

9-10 

Year  of  Birth 

11 

Gender 

1  =  Male 

2  -  Female 

12 

Type  Training 

1  =  FAA  Approved  School 

2  =  Military 

3  =  Other 

13 

Professional  Pilot 

1  =  yes 

2  =  no 

14 

Operations : 

1  =  coastal,  high  density 

2  =  coastal,  low  density 

3  =  flat,  high  density 

4  =  flat,  low  density 

5  -  mts ,  high  density 

6  =  mts,  low  density 

7  =  unknown 

15 

Field  Type: 

1  =  controlled 

2  =  uncontrolled 

Data  From  Recency  Program 

cc  16 

Flight  Number 

1  =  FI 

2  =  F2 

3  =  F3 

17-19 

Julian  Date 

20-23 

Total  Time  (prior  to  this  flight. 

24-27 

Total  Cessna  150  Time 

28-31 

Recency  (xx.xx  hrs/wk) 

32-35 

Landing  Recency  (xx.xx  lndgs/wk) 

36-38 

C-150  Recency  (x.xx  hrs/wk) 

39-41 

C-150  Lndg  Recency  (x.xx  lndgs/wk) 

whole  hrs) 


Data  From  Flight  Evaluation  Record:  (7  =  missing  data 

42  Instructor  Code:  1-6 

43-44  Quiz  Grade 

45  Overall  Grade  (round  all  grades  away  from  3) 

46-47  Flight  Planning 

48  A/C  Preflight 

49  Start,  Taxi,  Runup 
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50-51 

52 

53-54 

55-56 

57-58 

59-61 

62-63 

cc  64-65 

66-67 

68 

69-70 

71-72 

73-74 

75-76 

77-78 

79-80 


Take-off  and  Departure 

Slow  Flight 

Stalls 

VOR  Orientation  and  Tracking 
Simulated  Engine  Out 
Simulated  Loss  of  Horizon 
Pilotage  and  Dead  Reckoning 
Change  in  Flight  Plan 
Radio  Procedures 

Number  of  Landings  on  this  Test  Flight 
on  which  grades  were  given 
1 
2 

3 

4 

5 

6 
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Program 


Description 


01D 

02D 

03D 

05D 

08D 

02R 

03R 

01V 

07M 


Means ,  Standard  Deviations ,  Standard  Errors ,  Sample 
Sizes ,  Ranges 

2-way  Data  Plots 

Correlations 

Histograms 

Cross  Tabulations-Parametric  vs.  Non-Parametric 
Data 

Stepwise  Linear  Regression 

Multiple  Regression 

One-way  Analysis  of  Variance 

Stepwise  Discriminant  Function  Analysis 


BMP  Programs  (Ref.:  BMD-Biomedical  Computer  Programs,  W.J. 
Dizon,  ed.,  Univ.  of  California  Press,  Berkeley,  1970). 
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PILOTS  RECENCY  PROGRAM 


WRITE  ALL 
C -FOCAL, 1 969 

01.09  ERASE 

01.10  A  "SUBJECT  #",  SN»l,"- . -",  1,1,  "TOTAL  TIME  ",H 

01.30  A  "C-150  TIME  ",CH, ! 

01.50  A  "RECENCY  ",R4, ! 

01.70  A  "LNDG  RECENCY  ",RL» !, "C-150  RECENCY  ",CR, 1 

01.90  A  "C-150  LNDG  REC.'SRC, I,  !,"#  FLTS  IN  * 72", N,  ! 

01. 92  S  K-NJS  M-0. 

01.93  I  (K)  lo°4, 2. 1,1.94 
01.94  T  ! , "FLIGHT", %2.00,K 
01.95  G  5.01 

02.10  S  M-l. 

02.30  T  ! ,  f , f ."TEST  FLIGHT",* 2.00,M, 1,"  " 

02.40  D  5*0 

02.50  T  ! , "T OTAL  TIME  ",X5.00,H, 1 

02.60  T  "TOTAL  C-150  TIME  ",CH,  ! 

02.70  T  "RECENCY  "*X5*02,R4, I, "LANDING  RECENCY  ",RL, 

02.80  T  "C-150  RECENCY  ",CR, f ,"C-1 50  LNDG  RECENCY", RC, 1 , I 
02.81  I  (M-3.)  2.82,8.1,8.1 
02.82  A  "#  OF  INTERFLIGHTS", P, ! 

02.83  S  R» 1 • 

02.85  I  (P-R>  2.67,2.90,2.90 
02.87  S  M-M+1.*S  R-0.*G  2.3 

02.90  T  !, "INTERFLIGHT  #", *2 .00, R, f , "  " 

02*91  D  5*0 
02.93  S  R-R+l. 

02.95  G  2.85 

05.01  A  "  DATE",DF,"  DUR",HF, "  #  LNDGS",LF 

05.02  I  (M)  5.03,5.06,5.03 

05.03  I  <R>  5.06,5.12,5.06 

05.06  A  "C-1507",RE 

05.08  I  ( RE-0Y )  5.10,5.12,5.10 

05.10  S  OC-0.JS  OL»0.;G  5.16 

05*12  S  QC-HFJS  QL-LF 

05.16  I  (N-K)  5.22,5.42,5.22 

05.22  S  E«< (DF-DJ/28 ) * D  7.1 

05.23  I  CDF-D)  8.2,5.24,5.24 

05.24  S  H-H+DHJS  CH«CH+DC 

05.32  S  R4»(R4+.25*DH)*DT 

05.34  S  RL*<RL+.25*DL)«-DT 

05.38  S  CR«<CR+.25*DC>*DT!S  RC»(RC+.25*NC)*DT 
05.42  S  DH«HF*S  DL*LF*S  DC-QCJS  NC-QLIS  D**DF 
05.44  S  K*K- 1 . 

06.20  G  1.93 

07.10  S  DT»l/t  l+E+<<Et2>/2>-K<ET3>/6)M<Et4)/24>+<<E»5>/120>> 
08.10  T  f ,!,!,!, 2*QUIT 

08.20  T  !, "SEQUENCE  OF  FLIGHTS  ENTERED  REVERSED* BEGIN  AGAIN"* Q 
* 

PILOT  RECENCY  PROGRAM 
Appendix  P-2 


55 


r 

* 


V 

C -FOCAL* 1969 
01.10  E 

01.11  A  RJA  C;S  N=R*C 

01.20  F  I  ■  1  *  NJ  A  V<I)JS  T»V(I>+T 

01.21  T  !!! 

01.30  F  I«1*RJD  2 

01.40  F  I«1*CJD  3 
01.45  F  I*1*RJD  7 
01.50  f  i*i*r;d  4 
01.60  F  i»i*c;d  6 
01.70  T  X3*!!*"  GRAND  TOTAL"* T 

01.80  T  X6.03*!!*"  CHI  SQUARED"*  XI T  X3*"  DF"*<R-1>*CC-11* i! 

01.90  Q 

02.10  F  J*<(  1-1  )*C+1>*<<  I“1  )*C+Cvi  S  R(  I  >*R(  I >4>V(  J) 

03.10  f  j«0*r-i;s  ca>«c<n+v<i+jtc> 

04.10  T  X  3* ! ! * "ROW"* I  * "  ROW  TOTAL"*R(I> 

04.20  T  X4.04*"  X  OF  GR  TOT  IN  ROW  "*R<I>/T*fl 
04.30  F  J«1*C;D  5 

05.10  S  P-<RCI)*C(J)>/T;s  H(J)-V<M-1>#C+J> 

05.20  T  X2*"  COL"*  J* ! 

05.30  T  X3*"  ACTUAL"*H< J) * T  X6.03*"  EXPECTED"* P* I 

05.40  S  G< J)“P>t2/PiS  X«X+G<J> 

05.50  T  X6.03*"  CELL  CHI  SQ"*G(U>*! 

05.60  T  X4.04* "  X  COL"*HCJ)/CCJ>*"  X  ROV"*H(J)/R( I >* 1 

05.70  T  £4.04*"  X  TOTAL"* HC J) /T* !! 

06.10  T  X3* !* "COLUMN"* I*"  TOTAL"*CCI> 

06.20  T  X4.04*"  X  OF  GR  TOT  IN  COL  "*C(I)/T 

07.10  f  j«<a-i)*c+i>*<<i-i >*c+c>;t  X3*v<j> 

07.20  T  f 
* 


X2  CONTINGENCY  TABLE  ANALYSIS 
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0  100 


200  500  1000  2000  5000  10,000 

TOTAL  TIME  HOURS 


OVERALL  GRADE  VS.  TOTAL  TIME 
Figure  7 


YRS  SINCE  CERTIFICATION 


OVERALL  GRADE  VS.  YEARS  SINCE  CERTIFICATION 

Figure  8 
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— & - fc - ij - h 

TIME  IN  TYPE  (CESSNA  150)  HOURS 


OVERALL  GRADE  VS.  TIME  IN  TYPE  (CESSNA  150) 
Figure  9 


60 


OVERALL  GRADE 


Figure  ii 


FOUR  HIGHEST  AND  FOUR  LOWEST  SKILL  SCORES 

Figure  12 
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PER 

LANDING 


LANDING 

ACCURACY 

SCORE 


a  ~  1.3  SKILL  UNIT 


NO.  LANDINGS 


MEAN  LANDING  ACCURACY  SCORE  VS.  LANDING 

Figure  15 


UNITS  OF  SKILL  CHANGE 


